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Abstract 

Sensor data are the basis for performance and health assessment of most complex systems. Careful selection and 
implementation of sensors is critical to enable high fidelity system health assessment. A model-based procedure that 
systematically selects an optimal sensor suite for overall health assessment of a designated host system is described. 
This procedure, termed the Systematic Sensor Selection Strategy (S4), was developed at NASA John H. Glenn 
Research Center in order to enhance design phase planning and preparations for in-space propulsion health 
management systems (HMS). Information and capabilities required to utilize the S4 approach in support of design 
phase development of robust health diagnostics are outlined. A merit metric that quantifies diagnostic performance 
and overall risk reduction potential of individual sensor suites is introduced. The conceptual foundation for this 
merit metric is presented and the algorithmic organization of the S4 optimization process is described. 
Representative results from S4 analyses of a boost stage rocket engine previously under development as part of 
NASA’s Next Generation Launch Technology (NGLT) program are presented. 


Nomenclature 

DM Diagnostic Model 

DT Detection Threshold 

FMEA Failure Modes and Effects Analysis 

FT Appropriate Fault Response Family 

FM Diagnostic Model Indicated Fault Response Family 

GA Genetic Algorithm 

HMS Health Management System(s) 

LOX Liquid Oxygen 

LT Failure Threshold 

MMSE Minimum Mean Square Error 

NGLT Next Generation Launch Technology 

NP Nonlinear Polynomial Time 

RTM Real Time Model 

RP1 Type of hydrocarbon fuel 

SLI Space Launch Initiative 

S4 Systematic Sensor Selection Strategy 

A State space model state matrix 

B State space model input matrix 

C State space model output matrix 

D State space model influence or direct transmission matrix 

F Matrix of hardware influence functions 

O Observability matrix 

Q Observability Gramian matrix 
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h V ector of health parameters 

Sh Hardware parameter summation vector 

x Vector of state parameters 

y Vector of sensor derived measurements 

u Vector of independent inputs 

5 C Diagnostic model solution convergence criterion 

Sdet Measurement shift from normal indicating anomaly conditions 

5 P Distance to equal likelihood boundary 

n h Number of hardware parameters 

N m Number of fault modes 

N t Number of characteristic fault trajectories 

P Minimum probability most likely diagnosis is correct over all pairwise fault mode comparisons 

Prb Probability most likely diagnosis is correct for pairwise fault mode comparison 

R Risk reduction potential for timely diagnosis 

T Time to reach diagnostic threshold 

Tmin Minimum time to reach diagnostic threshold 

I. Introduction 

Health management systems (HMS) monitor and control the function of critical systems and components in 
order to ensure safe and efficient operation. The need for autonomous health management capability depends on the 
risk associated with system failure and the potential benefit of timely response to faulted or degraded operations. 
The criticality of the system and the intensity and/or hostility of the operating environment are important factors in 
assessing the need for HMS. Space systems in general, and especially man-rated systems, present an ideal 
application for realizing the benefits of health management technologies. 

Sensor data provide the foundation for performance and health assessment of most complex systems. Although 
opportunities for remote sensing of system function exist, the primary input for health diagnostics is generally 
provided by a network of sensors integrated within or in close proximity to the system of interest. Robust HMS 
require effective specification and placement of these sensors in order to support reliable and timely diagnoses over 
the range of potentially critical failure modes. Therefore effective sensor selection is an HMS enabling technology 
that supports agile control and planning for safe, productive, and cost effective system function. 

There is a substantial body of literature that treats the many, often competing, criteria that impact selection of 
sensors for system monitoring. This paper focuses on sensor selection for effective health diagnostics. It is organized 
in five parts. Background information on current selection strategies is presented in this section. While not an 
exhaustive review, the survey provides perspective of representative methods for diagnostic sensor selection. 
Subsequent sections present a conceptual foundation for sensor selection that facilitates health diagnostics and the 
logic structure of a new sensor selection process termed the Systematic Sensor Selection Strategy (S4). 
Representative results from an application of the S4 process to select diagnostic sensors for a large boost stage 
rocket engine system are then reported. The final section contains recommendations based on early experience with 
the S4 process. 

A. Traditional Approach 

There are many considerations that impact sensor selection for space propulsion health management applications 
(see ref. 1 for a list of typical criteria). Traditional approaches to measurement and sensor selection for space 
propulsion applications are generally heuristic and support performance measures, but may not produce an optimal 
suite for health diagnostics/prognostics. A typical first step in the selection process is to have the various engine 
component teams submit lists of desired measurements. These measurements are used to support engine 
development, model verification, and/or detection of physical limit (redline) violations. As the system design 
matures, the component teams supply more detailed specifications such as measurement ranges, response 
requirements, etc. This information along with reliability requirements is used to determine the type and number of 
sensors needed at each engine location. The compiled list is then separated into categories associated with 
measurement use, e.g. ground test, flight, etc. Measurements are assigned a priority, with the highest priority given 
to measurements required for engine control. The list is often condensed as the design matures due to factors such as 
accessibility, cable routing, or reduced need. A maximum number of sensors is then determined based on 
storage/transmission capability, cost, and other considerations, which may include arbitrary limits. The component 
teams and chief engineer then negotiate until the final suite is selected. 
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Traditional approaches to sensor selection draw heavily upon domain expertise. They do not generally utilize a 
consistent quantitative method to assess the implications of choices on the diagnostic capability that enables 
effective health management. The absence of an accepted quantitative assessment methodology is a technology gap 
that must be closed to achieve the ambitious objectives of future space missions. Methods for assessing diagnostic 
capability are available, and specific procedures are discussed in the sections that follow. 

B. Linear System Measures 

A well-developed theory and an extensive array of computational tools are available for solution of linear system 
problems and estimation of solution error bounds (see ref. 2). If health diagnostics for the system of interest can be 
effectively modeled in terms of a linear algebraic system, mature error estimation tools are available to guide sensor 
selection. A number of linear algebraic influence models (see refs. 3 to 8) have been developed for estimation of 
propulsion system health parameters. These models generally take the matrix form 


y = [d„ 


d £ ] 



(1) 


where D h is the hardware parameter influence matrix, D c is the matrix of control input influences, u h is the vector of 
hardware parameter deviations from a nominal baseline, u c is the vector of control input changes from a baseline 
command state, and y is the vector of measurement deviations from normal operating conditions. Knowledge of the 
control state u c and full column rank of the influence matrix D h are the criteria for detectability of any fault state that 
can be represented as a linear combination of hardware parameter deviations. Fault discrimination can be 
characterized by the condition number of the influence matrix D h . The condition number provides an indication of 
potential hardware state estimation error and its relation to measurement uncertainty. Large values increase state 
estimation uncertainty and impede clear and timely response to faults. Systematic selection of sensors to improve the 
condition estimate is a natural row partitioning process that provides quantitative metrics of diagnostic capability in 
terms of solution error bounds. Mathioudakis and Kamboukos (ref. 9) describe a sensor selection process based on 
minimization of the influence Jacobian condition number. Butas et al. (ref. 8) use a heuristic procedure for row 
partitioning of equation (1) to achieve influence matrix condition estimates consistent with accurate diagnostics. 

Although li sc tli I in many applications, sensor selection based on influence model condition estimates has several 
limitations. A linear influence model is inherently range limited for systems with appreciable nonlinear response 
characteristics. It does not explicitly consider sensor/system response dynamics nor does it discriminate hardware 
parameter solutions based on probability of occurrence over the range of combinations representing potential fault 
states. Good discrimination of hardware state solutions in regions with low probability of occurrence obviously 
contributes little to diagnostic effectiveness. 

State space models are a natural extension of influence methods commonly used to represent the behavior of 
physical systems. State space models can be represented by the general matrix relation 


X 


f(x,u) 

_y_ 


_g(x,u) 


(2) 


where x(t) is the state vector composed of variables needed to describe system behavior, x is the time derivative of 
x, and f and g are vector functions of the state and input variables (see ref. 10). Linear state space models are a 
simplified first order approximation of the general relations in equation (2) often used to approximate the behavior 
of systems (see refs. 1 1 and 12). Linear state space models can be expressed in the matrix form 


X 


"A 

B 

X 

y_ 


c 

D 

u 


where A, B, C, and D are the state, input, output, and direct transmission matrices respectively. A system described 
by equation (3) is said to be state observable if, for any time ti>0, the initial state x 0 = x(t = 0) can be determined 
from the time history of the input u(t), and the output y(t) in the interval [0,ti]. A linear time invariant system is said 
to be state detectable if all unstable modes are state observable (ref. 10). For such a system, state detectability is 
assured if and only if the observability matrix O given by the relation 
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O = [c CA ... CA n_1 ] T 


( 4 ) 


has full column rank. 

Fault detectability is of course necessary for effective health diagnostics; however, it does not characterize the 
timeliness of detection or the quality of fault discrimination. A number of scalar functions of the observability 
Gramian have been proposed to characterize the degree or quality of observability (refs. 13 to 15). The Gramian 
matrix Q is defined by the relation 


Q = Je AT ‘C T Ce At dt (5) 

o 

A clear description of the Gramian matrix and its role in state vector determination is provided by van den Berg 
(ref. 15). Dochain et al. (ref. 14) use the condition number of the Gramian to determine the most observable system 
model, and van den Berg describes several related Gramian metrics. Selection of the measurement variables y, 
together with the associated output matrix C, to optimize a Gramian fault discrimination metric provides a 
quantitative basis for systematic sensor selection. 

Although state space models are an effective means of characterizing system dynamics, they retain the range 
restrictions inherent with linear approximation of nonlinear system response. In addition, sensor selection strategies 
based on Gramian metrics do not explicitly consider the distribution of fault occurrences or severity over the range 
of potential state solutions x. For example, preferred fault directionality is an implicit fault distribution constraint 
that is not considered in Gramian metrics. As previously noted, good fault discrimination in state space regions with 
low probability of occurrence contributes little to diagnostic value. 

In practical applications, measurements are corrupted by random noise, and the system itself may be subjected to 
random disturbances. Therefore, individual state variables of a dynamic system can seldom be determined exactly 
from direct measurements. They must be estimated from noisy observations. The basic Kalman filter is a linear 
system estimator used when processes and measurements contain significant random components. Kalman filtering 
has been the subject of extensive research and application. If all random components are Gaussian, the Kalman filter 
estimator can be shown to be optimal in the minimum- mean-square-error (MMSE) sense (refs. 19 to 21). Extended 
Kalman filter strategies that approximate nonlinear system dynamics, albeit with additional complexity, are also 
available (refs. 19 to 21). Kalman filtering is especially useful for off-line diagnostics and error analysis studies. 
With continuing improvement in computational speed, Kalman filtering has increased potential for real-time 
diagnostics depending on response time requirements. 

Sensor selection based on minimizing the variance of health parameter estimation error from a state estimation 
Kalman filter has been considered in a number of studies (see refs. 16 to 20). Flowever use of error variance 
measures alone to guide sensor selection has certain shortcomings. Selection of measurement type and location can 
affect system dynamic response, potentially impacting timeliness of diagnosis as well as health parameter estimation 
error. In addition, models that describe fault propagation dynamics are generally immature in the early phases of 
system design. As a consequence, measurement distributions during fault sequences may be difficult to characterize 
with sufficient confidence for effective Kalman filtering. 

C. Targeted Fault Strategies 

System health diagnostics based on physical models associate fault conditions with the deviation of model health 
parameters from normal values. Sensor suites admitting reliable determination of fault states over the full continuum 
of potential health parameter values are most desirable. Flowever realistic fault scenarios will in general occupy only 
a fraction of the full parameter space range. This observation suggests the use of strategies that target regions of 
highest fault probability and severity. Targeted fault selection strategies often utilize specialized criteria to identify 
the optimal sensor suite for health diagnostics. In a series of papers, Bhushan and Rengaswamy (refs. 22 to 24) 
investigate the problem of sensor location based on various fault observability and resolution criteria. Narasimhan et 
al. (ref. 25) present four measurement selection algorithms using a qualitative reasoning framework to assess fault 
detection and isolation for dynamic systems. A method for designing a cost optimal sensor system for a designated 
diagnosability level is presented by Spanache et al. (ref. 26). In this latter study, a system is said to be fully 
diagnosable with a given set of sensors if and only if (i) for any relevant combination of sensor readings, there is 
only one minimal diagnosis candidate, and (ii) all faults of the system belong to a candidate diagnosis for some 
sensor readings. For most physical systems, only partial diagnosability can be realistically achieved because 
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multiple faults may share the same approximate signature and/or some faults are not detectable before failure occurs. 
In a recent study, Yan (ref. 27) considers sensor placement based on discriminability analysis where two faults may 
be discriminated if they generate different measurement output signatures over some subset of the available sensors. 

D. Complexity Analysis 

The complexity of various sensor selection problems has been investigated. Problems associated with 
determination of sensor sets of minimum cardinality guaranteeing diagnosability, normality, or observability for 
discrete event systems are considered by Yoo and Lafortune (ref. 28). Each is shown to be a member of the class of 
problems termed NP-complete (ref. 29). Problems in this class are notoriously difficult to solve, suggesting that 
optimal diagnostic sensor selection for complex systems can be a formidable task. This result motivates 
consideration of special problem structures and heuristic strategies leading to efficient selection algorithms. To be 
useful, however, the specified sensor suite must optimize diagnostic performance regardless of selection process 
complexity. The absence of a universally accepted diagnostic performance measure presents a significant 
impediment to robust sensor selection. 


II. Conceptual Basis 

There are a variety of pertinent criteria for selection of sensors supporting system health diagnostics. The 
focused objective of this effort was to develop a sensor selection strategy supporting high fidelity, real-time, targeted 
fault diagnostics that maximizes operational risk reduction. More specifically, we wish to select sensors to minimize 
fault detection time and maximize fault source discrimination in order to maximize targeted fault risk reduction for 
the system of interest. 

To facilitate overall system design assuring safety and reliability, and to minimize potentially expensive system 
retrofits, selection of sensors for effective health diagnostics is properly a design phase function. This limits the 
availability of test data as a selection guide, especially for prototype systems, and suggests that selection of sensors 
must be supported by fault simulations and diagnostic models. Limited system experience does not support fault 
diagnostics based on changes in measurement data variances, spectral properties, and/or complex pattern 
characteristics. Because of design evolution and limited test data availability, use of mean shifted performance data 
for fault detection and discrimination is indicated. 

To establish foundation and motivation for the S4 process, it is useful to have a conceptual understanding of 
propagating fault manifestations, and to characterize the impact of sensor selection on fault detection and isolation. 
The following subsections provide a pertinent conceptual framework. 

A. Trajectories 

For fault conditions to be diagnosable, they must induce sufficiently large measurement deviations from normal. 
A representation of the states traversed during a specific fault development history will be referred to as a fault 
signature or trajectory. The fault trajectory refers to both the measurement space sequence and the causal hardware 
parameter sequence associated with fault development. The function of the diagnostic model (DM) is to predict the 
hardware parameter sequence consistent with the observed measurement space trajectory. An approximation to the 
observed measurement sequence may be recovered by inserting the hardware parameter trajectory predicted by the 
DM into the system performance model and generating a measurement trajectory consistent with the DM solution. 
Differences between the observed measurement trajectory and the measurement trajectory recovered using the DM 
solution are a measure of the fidelity of the diagnostic model. Regenerated measurement trajectories based on DM 
results for faults designated A and B together with the actual fault B trajectory are identified in the notional two- 
dimensional measurement space depicted in figure 1 . 

An accurate DM is essential for reliable fault isolation. Inaccuracies can also delay fault detection or lead to false 
alarms. Trajectory implications for fault diagnostics are treated more fully in the subsections that follow. 

B. Fault Detection 

Fault detection requires sufficient measurement deviation to discriminate an anomaly condition from normal 
state variation. A plant hardware fault cannot be reliably discriminated from a sensor fault if measurements from 
only a single sensor source exhibit an anomaly excursion. Therefore, measurements from multiple sensor sources 
must exceed a prescribed threshold limit for reliable plant hardware fault detection. The minimum measurement 
deviation level for reliable fault detection within defined false alarm limits is designated Sdet- This is termed the 
detection threshold limit. In figure 1, the detectable fault zone is simply the measurement space region in which the 
output from both displayed measurement space sensors has exceeded the detection threshold limit. The boundary of 
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Figure 1. — Notional measurement space with well-defined fault trajectories. 


the detectable fault zone is termed the faidt detection threshold. A fault is initially detectable when its level is 
sufficient to reach a measurement trajectory location that intersects the detection threshold. 

C. Fault Isolation 

Once a fault condition has been detected, the DM attempts to isolate the fault source and provide a measure of 
fault severity. A fault is deemed to be isolated when the recovered DM solution trajectory point closest to the 
observed measurement data point is within a defined convergence threshold limit 5 C for one and only one fault. A 
fault is correctly diagnosed if the isolated DM solution corresponds to the true input fault type. 

The selection of sensors affects the proximity of the individual fault trajectories and hence the fidelity of fault 
discrimination. Closer measurement space trajectories are associated with less effective fault discrimination. For 
faults that may induce measurement shifts in opposed directions from the normal operating state (faults with linear 
bi-directional measurement trajectories), orthogonal trajectories are ideal for reliable fault discrimination. For a fast 
response system well represented by an influence model of the form given in equation (1), orthogonal trajectories 
correspond to minimal hardware influence matrix condition number and lower hardware parameter solution error 
bounds in the traditional sense (ref. 2). Flowever, component degradation and fault induced measurement trajectories 
are generally nonlinear and/or unidirectional (induce measurement shifts in only one direction). In such cases, 
orthogonal trajectories may not be optimal and a more general measure of fault discrimination is needed. 

During a realistic faidt progression, measurement data exhibit random scatter due to system/sensor noise and 
uncharacterized fault dynamics as depicted conceptually in figure 2. This scatter obscures the faidt induced 
measurement trajectory and complicates the detection process. If the selected sensor suite admits closely aligned 
measurement trajectories for distinct faults, reliable fault discrimination may be delayed well beyond the detection 
point. This is clearly seen in figure 2 by examining the fault A data for sensor suite 1 and the fault B measurement 
data. An effective sensor suite separates fault induced measurement trajectories and provides good fault 
discrimination capability. This is clearly depicted in figure 2 by the large separation of the fault A data using sensor 
suite 2 and the fault B data near their respective detection points. 
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Figure 2. — Notional measurement space with fault trajectories from diagnostic 
model solutions, and actual data from propagating faults for 
two distinct sensor suites. 


DM predicted fault trajectories are also displayed figure 2. For diagnostic systems utilizing measurement data 
with significant random fluctuation, a natural measure of fault discrimination is the probability that the most likely 
solution fault mode is the true fault mode. Since the objective is to select sensors that support reliable real-time fault 
diagnosis, it is important to achieve good fault discrimination immediately upon fault detection. Therefore the 
probability that the most likely DM solution fault mode corresponds to the true fault mode at the detection threshold 
is an appropriate measure of real-time fault discrimination potential. One method of estimating this probability is to 
determine the equal likelihood threshold limit 5 P depicted in figure 1. This threshold parameter is defined as a 
measure of the proximity of the initial fault detection state to the equal likelihood boundary between the DM fault 
signature of the true fault and the most proximate DM signature of another targeted fault mode. 

D. Fault Families 

Sensor selection that is narrowly focused to optimize health diagnostics for a small group of well defined faults 
is unlikely to support robust health management for systems with a wide range of potential fault manifestations. 
Fault families defined by combinations of hardware parameters describe large arrays of potential fault trajectories. 
Conceptual measurement space representations of a pair of two parameter fault family ranges are depicted in figure 
3. A specific fault occurrence generates an individual trajectory within the appropriate fault family measurement 
space range. The distribution of realistic fault trajectories within a given family is generally unknown. Flowever, a 
conservative estimate of DM fault family discrimination can be determined by examining the most proximate family 
trajectories. In figure 3, the single parameter fault A1 trajectory is the most proximate family A member to fault 
family B. Data scatter about the mean fault A1 detection point defines the probability that a family A fault will be 
determined as most likely. The previously described equal likelihood threshold distance 8 P can be used to generate 
an estimate of this probability. 
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Figure 3. — Notional measurement space with distinct fault families 
defined in terms of hardware parameter combinations. 


III. Selection Process 

The Systematic Sensor Selection Strategy, referred to as S4, is described in this section. S4 is intended primarily 
for system design phase utilization. It is a model-based, targeted-fault process for selection of sensors supporting 
health diagnostics. S4 can be logically partitioned into the three major subdivisions displayed in figure 4: the 
knowledge base, the down-select iteration, and the final selection analysis. Each of these subdivisions is described 
below. 

A. Knowledge Base 

The inputs required for productive use of S4 consist of evolving system design information together with a 
condensed form of related systems experience focused on components with health implications. This information 
provides a foundation for construction of effective system models, and together with these models constitutes the 
knowledge base needed to support use of S4. Pertinent knowledge base components are listed below. 

1. Failure modes and effects analysis (FMEA) identifying system critical faults and providing initial 
indications of sensible fault signatures 

2. Risk assessments associated with critical fault modes targeted for health diagnosis 

3. Candidate sensors listed by location and type, including associated response characteristics 

4. Estimates of measurement output variances due to sensor noise and system effects for candidate sensors 

5. Remediation response families correlated to targeted fault modes and severity levels 

6. False alarm constraints 

7. Model(s) of normal system operation based on the experience base of related systems and the appropriate 
physics governing as designed system function 

8. Fault simulation model(s) correlating targeted fault modes to the system model hardware parameters whose 
deviations can be used to describe these modes and their severity 

It is important to note that knowledge base components evolve during the system design phase and hence diagnostic 
sensor selection should be considered an integral component of the overall system design iteration sequence. 
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Figure 4. — Systematic Sensor Selection Process Diagram. 


B. Down-Select Iteration 

The sensor suite down-selection is an iterative process for identifying a group of sensor suites that provide good 
fault detection and isolation for targeted fault trajectories. It is composed of three basic components: a system health 
diagnostic model, a sensor suite merit algorithm, and a sensor suite down-select algorithm. The function and 
interactions of each component are described below. 

1. Sensor Suite Merit Algorithm 

The merit algorithm assigns a diagnostic merit value to each candidate sensor suite based on detection speed, 
probability of correct fault remediation family identification at the fault detection threshold, and risk reduction 
measures. The hardware parameter groups that describe each fault mode and the remediation response family 
associated with each fault mode are knowledge base inputs. An individual measurement trajectory for a given fault 
mode is an output sequence that may be approximated by the system simulation model. It is generated by causal 
hardware parameter time evolution input to the system simulation model. The number of discrete fault trajectories 
considered in characterizing diagnostic capability for a targeted fault mode is dictated by the range and distribution 
of mode faults. System domain expertise and knowledge base information guide definition of mode characteristic 
hardware fault trajectories. At a minimum, a single parameter trajectory for each hardware parameter pertinent to the 
fault mode as well as the highest likelihood multiple hardware parameter trajectories associated with the fault mode 
should be considered. Using characteristic fault mode trajectories, the merit value associated with a specific sensor 
suite is constructed using the component definitions below. 

DTjj detection threshold state for true fault mode i trajectory j 
LT;j failure threshold state for true fault mode i trajectory j 

DM ;i j diagnostic model mode i solution state for DT n using measurements available with current sensor suite 
DM ki j diagnostic model mode k solution state for DT, using measurements available with current sensor suite 
FTjj appropriate fault response family for DTy 

FMijj fault response family associated with best mode i DM solution state for DTy using measurements 
available with current sensor suite 

FM k jj fault response family associated with best mode k DM solution state for DTy using measurements 
available with current sensor suite 
N m number of fault modes considered 

N Ti number of characteristic trajectories considered for fault mode i 
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Ry risk reduction potential allocated to timely diagnosis of fault mode i trajectory j 

N m Mi 

R t total risk reduction potential = Z Z R ;; 

i=l j=t 

T;j time to reach DT y using a given sensor suite 

Tminy minimum time to reach DT M using any sensor suite 

0 if LTy reached before DTy using the given sensor suite or FM;y ^ FTjj 
Prbtij ■ 1 if DTy reached before LTy using the given sensor suite, FM;y = FTy , and FM k y = FT;j 

B < 1 if DTy reached before LTy , B = probability FM;y = FTy more likely than FM k y ^ FT 

Py min (Prb kij ) 

modes k^i 

The overall merit value of a specific sensor suite is defined in terms of the parameters above. 


Merit = — ZZ R u p i 


IJ Tmin: 


( 6 ) 


The merit function identifies, i) the overall timeliness of fault condition detection as a fraction of the fastest possible 
detection time using any set of candidate sensors (terms Ty/Tminy), and ii) the quality of fault discrimination in 
terms of the minimum probability of correct remediation response family identification near the detection threshold 
(terms Py). The discrimination quality metric is based on pairwise fault mode likelihood comparisons. The product 
of these two terms is used to estimate the fraction of risk reduction potential that can be ideally realized at the fault 
detection threshold with a specific sensor suite. This is a useful metric for assessing real-time health monitoring 
potential of candidate sensor suites. The ideal merit value is 1.0, indicating greatest potential real-time risk reduction 
for targeted fault modes using available sensors. 

The merit evaluation algorithm accepts a variety of one-time inputs from the knowledge base including true 
trajectory characteristics for all targeted fault modes. It also receives diagnostic model solutions for targeted fault 
trajectories and proximate mode trajectories for each sensor suite under consideration. The merit algorithm supplies 
diagnostic merit values for each current sensor suite to the down-select algorithm described below. 

2. Down-Select Algorithm 

The combinatorial nature of the down-select iteration 
lends itself to the use of a genetic algorithm (GA), hence a 
GA was developed to perform down-selection of candidate 
sensor suites. At each GA iteration stage or generation, Parent 1 = 

candidate sensor suites in the current (input) population f 
compete to evolve suites with improved diagnostic | Parent2 = 
capability, and to pass desirable properties to subsequent ^ 

stages of iteration or generations. A detailed description of l Child 1 = 

typical GA operations is given in Goldberg (ref. 30). \ \ 

Encoding the sensor selection problem in a GA format 1/ Child2 = 

is a simple process. Each individual member of a 
population is referred to as a chromosome and represents a Figure 5. Single point crossover example, 
single candidate sensor suite. Each chromosome is 

composed of bits called genes associated with individual candidate sensors. Binary encoding of the gene indicates if 
the particular sensor associated with the gene is in the sensor suite represented by the chromosome. A chromosome 
mask may be employed to force either selection or omission of particular sensors as desired. 

Each chromosome in the input population has an assigned merit value received as input from the merit 
algorithm. To choose output sensor suites passed to the next generation, roulette-wheel selection was employed (ref. 
30). With roulette-wheel selection, sensor suites with higher relative merit values have an increased probability of 
being selected as parents. Once parent suites are selected, single point crossover is employed to construct offspring 
suites for the next generation. The crossover point is selected randomly and genes are copied from the parents as 
depicted in figure 5. Elitism is used to advance the best sensor suites to the next generation without modification. 

Mutation or random bit-flipping, is employed during parent-to-child gene copying for its diversifying effects. As 
the generations progress, the population will increasingly converge to a group of similar sensor suites providing 
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good diagnostic capability. Increasing the probability of mutation with each generation aids in increasing the 
diversity of the population. Preserving diversity assists the population in evolving away from local optima in the 
solution space. 

The basic steps of the GA employed in this study are given below. 

Step 1 Initialize — Input the initial population of sensors from the knowledge base and make it the current 
generation. The initial population is randomly generated. 

Step 2 Merit Function — Input the merit algorithm assigned value for each sensor suite in the current generation. 

Step 3 Elitism — Automatically advance 2 sensor suites with the highest merit value to the next generation. 

Step 4 Selection — Select two sensor suites using roulette -wheel selection. 

Step 5 Crossover — Determine if crossover occurs. 

Step 5a No Crossover — Both parents advance to the next generation without modification. 

Step 5b Crossover — Apply single point crossover with the possibility of mutation as each gene is copied. 
Step 6 Repeat — Return to Step 4 until next generation is populated. 

Step 7 Repeat — Set the current generation equal to the newly formed next generation. 

Step 8 Output — Send current generation of sensor suites to the DM to reinitiate the merit assignment process. 

Return to Step 2 until target number of generations is reached. 

3. System Diagnostic Model 

Any procedure that is capable of assigning time indexed hardware health parameter values based on current and 
past measurement states is a candidate system diagnostic model. Since diagnostic speed is a critical issue for real- 
time applications, an inverse model was employed to estimate hardware state conditions at each time increment. The 
most straightforward type of inverse model determines hardware parameter deviations u h from normal by solving a 
set of system approximating relations of the form 


y = g(u h , u c ) (7) 

where y is the vector of diagnostic system measurements available for a given sensor suite. The control state vector 
u c and diagnostic measurement state vector y are treated as inputs and some type of general nonlinear equation 
solver is used to determine hardware state conditions u h at each sampled time slice. For S4 purposes, a modified 
Levinberg-Marquardt procedure (ref. 31) was used as the DM solver 

If the output measurement variation from normal is assumed to be a summation of control effects and 
independent single parameter fault effects, a considerable simplification of equation (7) may be achieved. 

y = F(u h ) s h + g(u c ) (8) 

In this expression, the function matrix F(u h ) is composed of elements fy(Uhj) that provide the contribution of each 
hardware parameter Uhj to each individual measurement y;. The n h x 1 summation vector s h = [1 1 . . . 1] T effects 
superposition of the individual hardware parameter contributions. A planned sequence of system fault simulations is 
required to support development of the functional relations in either of equations (7) or (8). Of course the support 
sequence is much simplified if the summation form of equation (8) provides an adequate representation of targeted 
fault manifestations. This would be the case if for instance targeted fault modes are primarily single parameter 
modes. 

The standard inverse model approach trades increased fidelity available from more detailed dynamic response 
models for improved diagnostic speed afforded by assuming instantaneous or matched propagation characteristics. 
For systems with well-matched measurement time constants or slow fault propagation rates, this trade is most 
favorable for effective real-time diagnostics. 

C. Statistical Evaluation Algorithm 

Because effective real-time diagnostic models must address the trade between diagnostic fidelity and speed, it is 
important to challenge both criteria in selecting an optimal sensor suite. The down-select iteration provides a set of 
“good” sensor suites based on approximate risk coverage, and speed. However, diagnostic model simplifications, 
sensor/system noise characteristics, and variations in fault manifestation dynamics from the assumed form for 
targeted fault trajectories provide sources of diagnostic performance uncertainty. The statistical evaluation algorithm 
is intended to provide a final robustness test for each down-selected sensor suite. The fault test case protocol may 
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include best estimates of measurement fluctuation due to i) sensor/system noise, ii) systematic variation in fault 
mode manifestation dynamics, and/or iii) random variations in fault manifestation. Once the test protocol is 
established, fault simulations are run to provide near detection point data for each test case. The ability of the 
diagnostic model to identify the correct response family using this data is assessed using the evaluation metric 
described in equation (6). The sensor suite that maximizes the merit metric is identified as the optimal sensor suite 
for health diagnostics. 


IV. Representative Results 

The S4 process described above was applied to identify optimal suites of flow path sensors for health diagnostics 
of rocket engine systems. Two systems developed through conceptual design by Boeing Rocketdyne as part of 
NASA’s Space Launch Initiative (SLI) and Next Generation Launch Technology (NGLT) programs served as test 
platforms for the S4 process. S4 analysis information and representative results for the RS-84 engine system are 
presented below. The RS-84 engine system was a large liquid fueled LOX-RP1 staged combustion engine originally 
intended for boost stage use in support of payload/vehicle orbital insertion. Its design vacuum thrust was in excess of 
one million pounds. 

The knowledge base for this application was obtained primarily from Space Shuttle Main Engine (SSME) data 
archives and Rocketdyne engine system component teams. A Real Time Model (RTM) of the RS-84 engine 
provided the basis of engine performance predictions and fault simulation cases used to support the S4 process, 
including construction of the RS-84 inverse model that served as the DM basis. General characteristics of Real Time 
Models for related system applications can be found in references 32 and 33. Only single parameter hardware faults 
and single source leaks were considered, hence equation (8) was used as the basic DM relation. System faults and 
risks defined by component teams were allocated to single parameter fault modes using a heuristic but logical 
procedure. The population of each generation considered in the GA was composed of 100 sensor suites. 

Information defining the scope of the S4 analysis for the RS-84 engine system is presented in table 1 below. The 
30 hardware faults included i) turbomachinery parameter faults, ii) duct, valve, and injector resistance or blockage 
faults, and iii) combustion chamber and nozzle performance faults. The 42 single source leak faults were specified 
by device and location. A total of 58 sensors was defined as the pool of candidates for health diagnostics. 

Summary results of the S4 analysis are presented in table 2. Leak faults were generally undetected prior to 
reaching a system failure point using the inverse model with candidate flow path sensors, typical system/sensor 
noise levels, and detection thresholds consistent with a low allowable false alarm rate. One way to address this 
problem is to invoke data filters that reduce noise levels and hence the detection threshold value. This is a viable 
option only if leak rate propagation is long relative to the filter interval. Data filtering experimentation was 
performed with considerable success relative to reduction in detection thresholds to facilitate leak detectability. 
However, no knowledge base information was available to characterize leak rate propagation rates and timeframes 
relative to failure onset or filtering intervals. Therefore sensor selection was performed targeting only the 30 single 
parameter hardware faults in the list of targeted faults. A complete inverse model diagnostic cycle, including 
inversions for all 30 hardware parameter faults, could be performed at a frequency greater than 20 Hz using a 
standard 800 megahertz Pentium processor. 

Control responses for various fault modes had not been formalized as of initial S4 analyses. Therefore control 
and maintenance response families had not been finalized and the DM discrimination requirement was not well 
defined. For initial S4 testing, individual hardware parameter response families were assumed and single parameter 
faults initiated only during steady-state operation were considered. Use of individual parameter specific response 
families is an extremely ambitious fault discrimination requirement, certainly more stringent than is likely to be 
required for effective health diagnostics. Using individual hardware parameter response families, an optimal merit 
index value of 0.62 was achieved as displayed in table 2. Most of the merit losses from the ideal value of 1.0 were, 
not surprisingly, due to fault discrimination issues rather than fault detection. The merit index value of 0.62 for the 
optimal sensor suite was considered reasonable considering the level of design definition as of the initial S4 analysis 
sequence. 

The general configuration of the optimal sensor suite is also identified in table 2. It is composed of 31 sensors 
used for health diagnostics, 6 of which are used to establish control and boundary conditions defining the normal 
engine operating state. 
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TABLE 2.— S4 OPTIMAL SENSOR 

TABLE L— RS-84 S4 ANALYSIS BASIS. SUITE FOR RS-84. 


Targeted Faults 

Hardware Faults (single param) 
Leak Faults (single source) 
Total Faults 

30 

42 

72 

Candidate Sensors (type+location) 


Flow Rate 

17 

Pressure 

22 

Shaft Speed 

4 

Temperature 

11 

Valve Position 

3 

Composite 

1 

Total Sensors 

58 


Optimal Diagnostic Sensor Suite 
Merit Evaluation (Hdwe Faults Only) 

Merit Index 

0.62 

Optimal Diagnostic Sensor Suite 

(Hardware Faults Only) 

Flow Rate 

3 

Pressure 

12 

Shaft Speed 

4 

Temperature 

10 

Valve Position 

2 

Total Sensors 

31 

(Total includes 6 boundary and control state input 

defining sensors) 




V. Summary and Recommendations 

The Systematic Sensor Selection Strategy described herein provides a logical procedure for quantifying the value of 
candidate sensor suites for targeted fault diagnostics based on criteria pertinent to real-time health management. 
These criteria include speed of detection, probability of correct fault source isolation, and overall risk reduction 
potential. The S4 process also provides a framework for organizing and utilizing an evolving system knowledge 
base, including system performance and simulation models, in the selection process. The sensor suite identified by 
the S4 process as optimal for health diagnostics is closely associated with the diagnostic model employed for fault 
identification. The diagnostic model must reflect the desired compromise of diagnostic fidelity and speed consistent 
with false alarm limits and potential remediation responses that maximize overall operational risk reduction. 

Based on experience with S4 development and general consideration of the role of sensor selection in effective 
system health management, the following recommendations are offered. 

1. Effective health management is enabled by the availability of sensor data that facilitates health diagnosis. 
Therefore systematic sensor selection for improved diagnostics should be an integral component of health 
management system development. 

2. To maximize potential risk reduction benefit, sensor selection for health diagnostics should be an integral 
system design phase activity that both evolves with and impacts host system design. 

3. The linkage between physical fault modes, the parameter based simulation of these modes, and the model 
based use of hardware parameter shifts for fault isolation should be strengthened to support more robust 
health diagnostics. 

4. Sensor faults are not explicitly considered by the current S4 process. Methods of expanding the process to 
support sensor selection for health management system data qualification/validation should be explored. 

5. Extensions of the standard inverse model to better consider system nonlinearity and fault dynamics are 
accessible. These extensions and other diagnostic models should be examined within the S4 framework to 
determine the best compromise of diagnostic fidelity and speed for specific applications. 

6. The S4 process may be used to quantitatively assess the impact of specific sensor losses during operation and 
potential sensor additions on diagnostic capability. This capability should be refined and systematically 
applied for long-term health management system development and maintenance. 

References 

1. Campbell, G., Johnson, J., Obleski, M., and Puening, R., “Rocket Engine Condition Monitoring System 

(RECMS),” Final Report — SHM Design Methodology, Pratt & Whitney MCR-92-5014, October 30, 1992. 

2. Golub, G. and Van Loan, C., Matrix Computations, 3rd Edition, The Johns Hopkins University Press, 

Baltimore, MD, 1996. 

3. Volponi, A., “Gas path analysis: an approach to engine diagnostics,” Symposium of Mechanical Failures 

Prevention Group, Gaithersburg, MD, 1982. 


NASA/TM— 2005-213955 


13 





4. Doel, D., “An Assessment of Weighted-Least-Squares-Based Gas Path Analysis,” Journal of Engineering for 
Gas Turbines and Power, vol. 116, 1994, pp. 366-373. 

5. Doel, D., “TEMPER — a Gas-Path Analyis Tool for Commercial Jet Engines,” Transactions of the ASME, vol. 
116, 1994, pp. 82-89. 

6. Mathioudakis, K., Stamatis, A., Berios, G., and Papailiou, K., “Jet Engine Fault Detection With Discrete 
Operating Points Gas Path Analysis,” Journal of Propulsion, vol. 7, 1991, pp. 1043-1048. 

7. Santi, L., and Butas, J., “Generalized Data Reduction Strategy for Rocket Engine Applications,” AIAA-2000- 
3306, 36th AIAA/ASME/SAE/ASEE Joint Propulsion Conference and Exhibit, July 2000. 

8. Butas, J., Meyer, C., Santi, L., and Sowers, T., “Rocket Engine Health Monitoring Using a Model-Based 
Approach,” AIAA-2001-3764, 37th AIAA/ASME/SAE/ASEE Joint Propulsion Conference and Exhibit, July 
2001 . 

9. Mathioudakis, K. and Kamboukos, P., “Assessment of the Effectiveness of Gas Path Diagnostic Schemes,” 
Proceedings of ASME Turbo Expo 2004 - Power for Land, Sea, and Air, Vienna, Austria, June 2004. 

10. Skogestad, S. and Postlethwaite, I., Multivariable Feedback Control, John Wiley & Sons, Ltd., Chichester, 
West Sussex, England, 1996. 

1 1. Kailath, T., Linear Systems, Prentice-Hall, Englewood Cliffs, New Jersey, 1980. 

12. Zhou, K., Doyle, J., and Glover, K., Robust and Optimal Control, Prentice-Hall, Upper Saddle Creek, 1996. 

13. Muller, P. and Weber, H., “Analysis and Optimization of Certain Qualities of Controllability and Observability 
for Linear Dynamical Systems, Automatica, vol. 8, 1972, pp. 237-246. 

14. Dochain, D., Tali-Manaar, N. and Babary, J., “On Modeling, Monitoring and Control of Fixed Bed Bioreactors, 
Computers and Chemical Engineering, vol. 21, 1997, pp. 1255-1266. 

15. van den Berg, F., Hoefsloot, H., Boelens, H., and Smilde, A., “Selection of Optimal Sensor Position in a 
Tubular Reactor Using Robust Degree of Observability Criteria,” Chemical Engineering Science, vol. 55, 2000, 
pp. 827-837. 

16. Chen, W. and Seinfeld, J., “Optimal Location of Process Measurements,” Chemical Engineering Science, 
vol. 32, 1975, pp. 1035-1049. 

17. Aidarous, S., Gevers, M., and Installe, M., “Optimal Sensors’ Allocation Strategies for a Class of Stochastic 
Distributed Systems,” International Journal of Control, vol. 22, 1975, pp. 197-213. 

18. Kumar, S. and Seinfeld, J., “Optimal Location of Measurements for Distributed Parameter Systems,” IEEE 
Transactions on Automatic Control, vol. AC-23, 1978, pp. 690-698. 

19. Grewal, M. and Andrews, A., Kalman filtering theory’ and practice, Prentice-Hall, Englewood Cliffs, New 
Jersey, 1993. 

20. Litt, J., “Sensor Placement for Aircraft Propulsion System Health Management,” NASA TM, in review. 

21. Brown, R. and Hwang, P., Introduction to Random Signals and Applied Kalman Filtering, 3rd Edition, John 
Wiley & Sons, New York, 1997. 

22. Bhushan, M. and Rengaswamy, R., “Design of Sensor Network Based on the Signed Directed Graph of 
the Process for Efficient Fault Diagnosis,” Industrial and Engineering Chemistry’ Research, vol. 39, 2000, 
pp. 999-1019. 

23. Bhushan, M. and Rengaswamy, R., “Comprehensive Design of Sensor Network for Chemical Plants Based on 
Reliability Criteria: I. Framework,” Industrial and Engineering Chemistry’ Research, vol. 41, no. 7, 2002, pp. 
1826-1839. 

24. Bhushan, M. and Rengaswamy, R., “Comprehensive Design of Sensor Network for Chemical Plants Based on 
Reliability Criteria: II. Applications,” Industrial and Engineering Chemistry Research, vol. 41, no. 7, 2002, pp. 
1840-1860. 

25. Narasimhan, S., Mosterman, P.J., and Biswas, G., “A Systematic Analysis of Measurement Selection 
Algorithms for Fault Isolation in Dynamic Systems,” Ninth Inti. Workshop on Principles of Diagnosis, Cape 
Cod, MA, May 1998, pp. 94-101. 

26. Spanache, S., Escobet, T., and Trave-Massuyes, L., “Sensor Placement Optimisation Using Genetic 
Algorithms,” Proceedings DX-2004, 15th International Workshop on Principles of Diagnosis, Carcassonne, 
France, June 2004. 

27. Yan, Y., “Sensor Placement and Diagnosability Analysis at Design Stage,” National Research Council of 
Canada, NRC 47160, 2004. 

28. Yoo, T.-S. and Lafortune, S., “NP-Completeness of Sensor Selection Problems Arising in Partially Observed 
Discrete-Event Systems,” IEEE Transactions on Automatic Control, vol. 47, 2002, pp. 1495-1499. 

29. Papadimitriou, C. and Steiglitz, K., Combinatorial Optimization: Algorithms and Complexity, Dover, New 
York, 1998. 


NASA/TM— 2005-213955 


14 



30. Goldberg, D., Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, 1989. 

31. More, J., “The Levenberg-Marquardt Algorithm, Implementation and Theory” in Numerical Analysis, G.A. 
Watson, editor. Lecture Notes in Mathematics 630, Springer- Verlag, 1977. 

32. Panossian, H., Nelson, R., Tran, H., Klop, J., Aguilar, R., Poladian, T., and Sinai, M., “Aerospike Engine 
Control System Design,” AIAA-1999-2935, 35th AIAA/ASME/SAE/ASEE Joint Propulsion Conference and 
Exhibit, June 1999. 

33. Aguilar, R. and Garcia, R., “Strategy for Artificially Intelligent Health Monitoring of a Large Reusable Rocket 
Engine,” ADIUS (Applied Dynamics Users Society), Incline Village, Nevada, June 1992. 


NASA/TM— 2005-213955 


15 



REPORT DOCUMENTATION PAGE 

Form Approved 
OMB No. 0704-0188 

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 
gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 
collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson 
Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503. 

1 . AGENCY USE ONLY (Leave blank) 

2. REPORT DATE 

September 2005 

3. REPORT TYPE AND DATES COVERED 

Technical Memorandum 

4. TITLE AND SUBTITLE 

5. FUNDING NUMBERS 


Optimal Sensor Selection for Health Monitoring Systems 


6. AUTHOR(S) 

L. Michael Santi, T. Shane Sowers, and Robert B. Aguilar 


WB S-22-7 94-20- 1 2 


7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

National Aeronautics and Space Administration 
John H. Glenn Research Center at Lewis Field 
Cleveland, Ohio 44135-3191 


8. PERFORMING ORGANIZATION 
REPORT NUMBER 


E- 15271 


9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

National Aeronautics and Space Administration 
Washington, DC 20546-0001 


10. SPONSORING/MONITORING 
AGENCY REPORT NUMBER 

NASA TM— 2005-213955 
A1AA-2005-4485 


11. SUPPLEMENTARY NOTES 

Prepared for the 4 1 st Joint Propulsion Conference and Exhibit cosponsored by the AI AA, ASME, S AE, and ASEE, Tucson Arizona, 
July 10-13, 2005. L. Michael Santi, Glenn Research Center (currently at Christian Brothers University, 650 E. Parkway S., 
Memphis, Tennessee 38104); T. Shane Sowers, Analex Corporation, 1100 Apollo Drive, Brook Park, Ohio 44142; and Robert B. 
Aguilar, Boeing Rocketdyne Propulsion and Power, 6633 Canoga Avenue, Canoga Park, California 91309. Responsible person, T. 
Shane Sowers, organization code RIC, 216-433-3405. 


12a. DISTRIBUTION/AVAILABILITY STATEMENT 

Unclassified - Unlimited 
Subject Categories: 20, 61, and 65 

Available electronically at http://gltrs.grc.nasa. gov 

This publication is available from the NASA Center for AeroSpace Information, 301-62 1-0390. 


12b. DISTRIBUTION CODE 


13. ABSTRACT (Maximum 200 words) 

Sensor data are the basis for performance and health assessment of most complex systems. Careful selection and 
implementation of sensors is critical to enable high fidelity system health assessment. A model-based procedure that 
systematically selects an optimal sensor suite for overall health assessment of a designated host system is described. This 
procedure, termed the Systematic Sensor Selection Strategy (S4), was developed at NASA John H. Glenn Research 
Center in order to enhance design phase planning and preparations for in-space propulsion health management systems 
(HMS). Information and capabilities required to utilize the S4 approach in support of design phase development of 
robust health diagnostics are outlined. A merit metric that quantifies diagnostic performance and overall risk reduction 
potential of individual sensor suites is introduced. The conceptual foundation for this merit metric is presented and the 
algorithmic organization of the S4 optimization process is described. Representative results from S4 analyses of a boost 
stage rocket engine previously under development as part of NASA’s Next Generation Launch Technology (NGLT) 
program are presented. 


14. SUBJECT TERMS 

Systems health monitoring; Qualification; Diagnosis; Program verification; Propulsion; 
False alarms; Error analysis; Performance tests; Simulation 


15. NUMBER OF PAGES 

21 


16. PRICE CODE 


17. SECURITY CLASSIFICATION 
OF REPORT 

Unclassified 


18. SECURITY CLASSIFICATION 
OF THIS PAGE 

Unclassified 


19. SECURITY CLASSIFICATION 
OF ABSTRACT 

Unclassified 


20. LIMITATION OF ABSTRACT 


NSN 7540-01-280-5500 


Standard Form 298 (Rev. 2-89) 
Prescribed by ANSI Std. Z39-18 
298-102 



